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Abstract. An infinite binary sequence has randomness rate at least a if, for almost every 
n, the Kolmogorov complexity of its prefix of length n is at least an. It is known that 
for every rational a 6 (0, 1), on one hand, there exists sequences with randomness rate a 
that can not be effectively transformed into a sequence with randomness rate higher than 
a and, on the other hand, any two independent sequences with randomness rate a can be 
transformed into a sequence with randomness rate higher than a. We show that the latter 
result holds even if the two input sequences have linear dependency (which, informally 
speaking, means that all prefixes of length n of the two sequences have in common a 
constant fraction of their information). The similar problem is studied for finite strings. 
It is shown that from any two strings with sufficiently large Kolmogorov complexity and 
sufficiently small dependence, one can effectively construct a string that is random even 
conditioned by any one of the input strings. 



1. Introduction 

The randomness rate of an object is the ratio between the information in the object and 
its length. An informal principle states that no reasonable transformation can guarantee an 
increase of the randomness rate. The principle has different instantiations depending on the 
meaning of "object", "information," and "reasonable transformation." For example, if / is 
a mapping of the set of n-bit strings to the set of m-bit strings, then there is a distribution X 
on the set of n-bit strings with Shannon entropy n/2 (i.e., the randomness rate of X is 1/2) 
and the Shannon entropy of f(X) is < m/2 (i.e., the randomness rate of f(X) is < 1/2). 
Thus no transformation / guarantees that its output has a randomness rate higher than that 
of its input. The case of infinite binary sequences (in short, sequences) is very interesting 
and has been recently the subject of intensive research. We say that a sequence x has 
randomness rate at least a if K(x \n) > a-n, for all sufficiently large n. Here, x \n is the prefix 
of length n of x and K(-) is the Kolmogorov complexity. A related notion is that of effective 
Hausdorff dimension of a sequence x, defined as: dim(x) = liminf K(x|"n)/n. Reiman and 
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Terwijn [Rei04j have asked whether for any sequence x with dim(x) = 1/2 there exists 
an effective transformation (formally, a Turing reduction) / such that dim(f(x)) > 1/2. 
Initially, some partial negative results have been obtained for transformations / with certain 
restrictions. Reimann and Terwijn [Rei04[ Th 3.10] have shown that the answer is NO if 
we require that / is a many-one reduction. This result has been extended by Nies and 
Reimann [NR06] to wtt-reductions. Bienvenu, Doty, and Stephan [BDS07] have obtained 
an impossibility result for the general case of Turing reductions, which, however, is valid only 
for uniform transformations. More precisely, building on the result of Nies and Reimann, 
they have shown that for all constants c\ and C2, with < c\ < C2 < 1, there is no 
Turing reduction / such that for any sequence x with dim(x) > ci has the property that 
dim(f(x)) > C2- In other words, loosely speaking, no effective uniform transformation is able 
to raise the randomness rate from c\ to C2- Finally, very recently, Miller [Mil08j has fully 
solved the original question, by constructing a sequence x with dim(x) = 1/2 such that, for 
any Turing reduction /, dim(f(x)) < 1/2 (or f(x) does not exist). 

On the other hand, Zimand [Zim08] has shown that it is possible to increase the ran- 
domness rate if the input consists of two sequences that enjoy a certain type of independence. 
Namely, we say that two sequences x and y are finitary- independent^ if for all n and m, 

K(x\n y\m) > K{x\n) + K(y \m) — 0(max(logn, logm)). (1-1) 

In [Zim08| . it is shown that for any constant < r < 1, there is a Turing reduction / such 
that, for any finitary-independent sequences x and y, both with randomness rate > r, it 
holds that f(x, y) has randomness rate arbitrarily close to 1 (in particular, dim(f(x, y)) = 1). 
Moreover / is a truth-table reduction and also / is uniform in r. 

To summarize, if we start with one source, it is impossible to effectively increase the 
randomness rate, while if we start with two finitary-independent sequences it is possible to 
increase the randomness rate to close to 1 in a uniform and truth-table manner. 

It is clear that the independence requirement plays an important role in the positive 
result. Since independence can be quantified, it is interesting to see what level of indepen- 
dence is needed for a positive result. 

For a function d : N — > R, we say that strings u and v have dependency d if K(u) + 
K(v) — K(uv) < d(max(\u\, \v\)); we say that two sequences x and y have dependency d if, 
for every n and m sufficiently large, the strings x \n and y \m have dependency d. With this 
terminology, sequences x and y are finitary-independent if they have dependency c • logn, 
for some positive constant c. 

The question becomes: How large can d be so that an effective increase of the random- 
ness rate is possible from two sequences with dependency d? Miller's result shows that this 
is impossible for dependency d(n) = n, while the result in [Zim08] shows that this is pos- 
sible for dependency d(n) = c ■ logn. In fact, [Zim08] shows that, for certain combinations 
of parameters, an effective increase is possible even for dependency d(n) = n a , for some 
< a < 1. More precisely, it is shown that for any r > and 5 > 0, there exists < a < 1 
and a truth-table reduction / such that for any sequences x and y that have randomness 
rate r and dependency d(n) = n a , it holds that f(x,y) has randomness rate 1 — 5. 

hn |Zim08] , such sequences are called independent. The paper [CZ08] examines thoroughly the concept 
of algorithmic independence for sequences and introduces besides nnitary-independence, a stronger concept 
which is called independence. We adopt here the terminology from CZ08 . 
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In this paper, we improve the above result and show that one can effectively increase 
the randomness rate even for two input sources that have linear dependency. More formally, 
our result is: 

(1) We show that for every < r < 1 and 5 > 0, there exist < a < 1 and a truth- 
table reduction / such that for any sequences x and y with randomness rate r and 
dependency d(n) = an, the sequence f(x,y) has randomness rate > (1 — 5). 

We also study the finite version of the problem, when the input consists of strings. 
Similarly to the infinite case, our interest is in determining how many input strings and 
what level of dependency are necessary in order to exist an effective procedure that extracts 
Kolmogorov complexity. Vereshchagin and Vyugin [VV02, Th. 4] have shown that one input 
string is not enough. They construct a string x so that any shorter string that has small 
Kolmogorov complexity conditioned by x (in particular any string effectively constructed 
from x) has small Kolmogorov complexity unconditionally. On the other hand, Fortnow, 
Hitchcock, Pavan, Vinodchandran and Wang [FHP + 06] show that an input consisting of 
several independent strings can accomplish the task, when the number of strings in the input 
depends on the complexity of the strings. Formally, they show that, for any a there exists a 
constant I and a polynomial-time procedure that from an input consisting of £ n-bit strings 
xi, . . . ,xg, each with Kolmogorov complexity at least an, constructs an n-bit string with 
Kolmogorov complexity y n — dep(xi, . . . , xe) (dep(xi, . . . , Xf) = YU=1 K( x i) ~ K( x i ■ ■ ■ x l) 
and >z means that the inequality holds within an error of 0(log n)). In view of Vereshchagin- 
Vyugin result, the question is whether effective extraction of Kolmogorov complexity is 
possible from two input strings. We have two results in this regard: 

(2) We show that if strings x and y of length n have dependency an and complexity an, 
then it is possible to effectively construct a string of length ~ 2a ■ n and complexity 
y (2<T — a) • n, where ~ (y) means that the equality (resp., the inequality) is within 
an error of O(logra). The construction is uniform in x,y,a,a. Note, however, that 
unlike the procedure from [FHP + 06] . the construction does not run in polynomial 
time. 

(3) Our second result shows that from strings x and y, with sufficiently large complexity 
and sufficiently small dependency, it is possible to construct a string z that has 
large complexity even conditioned by any of the input strings. More precisely if 
x and y are strings of length n that have complexity s(n) and dependency a(n), 
then it is possible to effectively construct a string of length m ~ s(n)/2 such that 
K(z | x) y m — a(n) and K(z \ y) y m — a(n). The construction is uniform 
in x,y, s{n),a(n). This improves a result from [CZ08.J, where the input consists of 
three strings x\, xi, X3 and the construction produces a string z with large K(z \ Xi), 
i = 1,2,3. 

Effective procedures that extract the Kolmogorov complexity of strings are related to ran- 
domness extractors. These are objects of major interest in computational complexity and 
there is a long and very active line of research dedicated to them. A randomness extractor 
is a procedure (which, ideally, runs in polynomial time) that improves the quality of a de- 
fective source of randomness. A source of randomness is modeled by a distribution X on 
{0, l} n , for some n, and its quality is modeled by the min-entropy of X {X has min-entropy 
k if 2~ k is the largest probability that X assigns to any string in {0, l} n ). The distribution 
X is defective if its min-entropy is less than n, and is perfect if its min-entropy is equal to 
n, which implies that X is the uniform distribution on {0, l} n . In many applications, it is 
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desirable to transform a defective distribution X into a distribution X' on a set of shorter 
strings which is close to the uniform distribution. Such a transformation is called a ran- 
domness extractor. Randomness extraction is not possible from a single source [SV86j . but 
it is possible from two or more sources |Vaz87] . Consequently, the research has focused on 
two types of extractors, seeded extractors and multi-source extractors. A seeded extractor 
extracts randomness from two independent distributions X and Y, where X is defective 
and defined on the set of ra-bit strings and Y is perfect and defined on the set of cZ-bit 
strings, with d much shorter than n (typically d = O(logra)). A /c-multisource extractor 
takes as input k defective distributions on the set of n-bit strings. For k = 2, the best 
multisource extractors are (a) the extractor given by Raz [Raz05] with one source having 
min-entropy ((1/2) + a)n (for some small a) and the second source having min-entropy 
polylog(n), and (b) the extractor given by Bourgain [Bou05] with both sources having min- 
entropy ((1/2) — a)n (for some small a). There is a clear analogy between randomness 
extractors and procedures that extract Kolmogorov complexity. In particular, the reader 
may compare results (2) and (3) discussed above with existing 2-multisource extractors, but 
we emphasize that there is a major difference in that extractors run in polynomial time, 
while the procedures in (2) and (3) are only in EXPSPACE. On the other hand, results (2) 
and (3) suggest that it might be possible to construct multisource extractors with sources 
having a certain level of dependence and/or with the output being random even conditioned 
by one of the sources. 

A few words about the proof technique. At the highest level, our method follows the 
structure of the proofs in [Zi m08| . One key idea is taken from Fortnow et al. [FHP + 06] . who 
showed that a multisource extractor also extracts Kolmogorov complexity. Since multisource 
extractors with the parameters that are needed here are not known to exist, we construct a 
combinatorial object, called a balanced table, that is similar with a 2-multisource extractor. 
A balanced table is a 2-dimensional N x N table with each entry having one of M colors 
such that in each sufficiently large subrectangle all the colors appear approximately the 
same number of times (see Definition 12. 2\i . Using the probabilistic method, we show the 
existence of balanced tables with appropriate parameters. It follows that such tables can 
be effectively constructed using exhaustive search. Next, using arguments similiar to those 
m jFHP+06] . we show that if x and y have sufficiently large complexity and sufficiently 
small dependence, then the color of the entry in row x and column y of the table has large 
complexity. These ideas are sufficient to establish result (2) (Theorem 13. ip . Results (1) 
(Theorem 14. ip and (3) (Theorem I3.2[) require non-trivial technical refinements of the basic 
method which are explained in the respective proofs. 

2. Preliminaries 
2.1. Notation 

We work over the binary alphabet {0, 1}. A string is an element of {0, 1}* and a 
sequence is an element of {0, 1}°°. If a; is a string, |x| denotes its length. If x is a string or a 
sequence and n £ N, x\n denotes the prefix of x of length n. The cardinality of a finite set 
A is denoted \A\. For n € N, [n] denotes the set {1, 2, . . . , n}. Let M be a standard Turing 
machine. For any string x, define the (plain) Kolmogorov complexity of x with respect to 
M, as 

Km{x) = min{|p| | M(p) = x}. 
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There is a universal Turing machine U such that for every machine M there is a constant 
c such that for all x, 

Ku(x) < K M (x) + c. (2.1) 
We fix such a universal machine U and dropping the subscript, we let K(x) denote the 
Kolmogorov complexity of x with respect to U. For the concept of conditional Komogorov 
complexity, the underlying machine is a Turing machine that in addition to the read/work 
tape which in the initial state contains the input p, has a second tape containing initially a 
string y, which is called the conditioning information. Given such a machine M, we define 
the Kolmogorov complexity of x conditioned by y with respect to M as 

Km{x I y) = min{|p| | M(p,y) = x}. 

Similarly to the above, there exist universal machines of this type and they satisfy the rela- 
tion similar to Equation EJ, but for conditional complexity. We fix such a universal machine 
U, and dropping the subscript U, we let K(x \ y) denote the Kolmogorov complexity of x 
conditioned by y with respect to U. 

Let a 6 [0,1]. A sequence x has randomness rate at least a if K(x{\ : n)) > a ■ n, for 
almost every n (i.e., the set of n's violating the inequality is finite). 

The procedures that we design for extracting the Kolmogorov complexity of strings 
or sequences are either computable functions (in the case of strings) or Turing reductions 
(in the case of sequences). In our result, the Turing reduction is also uniform in two 
parameters r and a. Formally, such a Turing reduction / is represented by a two-oracle 
Turing machine Mf. The machine Mf has access to two oracles x and y, which are binary 
sequences. When Mf makes the query "n-th bit of first oracle?" ("n-th bit of second 
oracle?"), the machine obtains x(n) (respectively, y(n)). On input (r, a, l n ), where r and 
a are rational numbers (given in some canonical representation), Mf outputs one bit. We 
say that f(x,y,r,a,) = z £ {0,1}°°, if for all n, Mf on input (r, a, l n ) and working with 
oracles x and y halts and outputs z(n). In case the machine Mf halts on all inputs and 
with all oracles, we say that / is a truth-table reduction. 

2.2. Limited Independence 

Definition 2.1. (a) The dependency of two strings x and y is dep(x,y) = K(x) + 
K(y)-K{xy). 

(b) Let d : N — * K. We say that strings x and y have dependency at most d(n) if 
dep(x,y) < d(max(|x|, \y\)). 

(c) Let d : N — > R. We say that sequences x and y have dependency at most d(n), if 
for every natural numbers n and m, the strings x\n and y\m have dependency at 
most d(n). 

2.3. Balanced Tables 

Let N and M be positive integers. An (N,M) table is a function T : [N] x [N] [M]. 
It is convenient to view it as a two dimensional table with N rows and iV columns where 
each entry has a color from the set [M]. If B\, B2 are subsets of [N], the B\ x B2 rectangle 
of table T is the part of T comprised of the rows in B\ and the columns in Bi- 
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Definition 2.2. Let T : [N] x [N] -» [M] be an (JV, M) table and S < N and D < M 
be two positive integers. We say that the table is (5, D)-balanced if for every set A C [M] 
with = M/D and for every sets S a C [N],B 2 C [AT] with |S X | > 5, |J3 a | > 5, 

|T~V) n (Bx x B 2 )\ < 2 ■ • \B X x B 2 |. 

The above definition states that in any I?i x i?2 rectangle of T and for any set A of 
colors of size M/D, the fraction of occurences of colors in A is bounded by 2 • |A|/M. 

Lemma 2.3. Suppose S 2 > 3M + 3MlnD + 6ST> + GST* \n(N/S). Then there exists a 
table T: [N] X [iV] -> [M] f/iai is (S, D) -balanced. 

Proof. The proof is by the probabilistic method. We color the N-by-N table selecting for 
each entry independently at random a color from [M]. Let us fix A C [M] with \ A\ = M/D, 
B\ C [N] with | Si | = S" and B 2 C [iV] with |2?2| = 5*. Note that it is enough to prove the 
assertion for sets B\ and B 2 of size exactly S. By the Chernoff bounds, 

/ number of ^-colored cells in B t x B 2 \A\\ (imA \/M)s^ _ ,-(i/(3D))s 2 
Fr ° b V 5 2 >Z Mj~ e ~ 6 

(2-2) 

The number of possibilities of choosing the set A as above is bounded by 

( M ) < (e • D) M > D = e M/D+(M/D)lnD_ ^ 



\M/Dj 

of chc 

(Tj 2 < (eN/S) 2S = e is+2S M N/s) {2A) 



The number of possibilities of choosing the sets B\ and B 2 as above is bounded by 

2 



The hypothesis ensures that the product of the upper bounds in Equations (|2.2p . (|2.3p . 
and (|2.4p is less than 1. It follows from the union bound that there exists an (S,D)- 
balanced table. ■ 

In our applications, ./V and M will be powers of two, N = 2 n , M = 2 m , and [N] is 
identified with {0, l} n and [M] is identified with {0, l} m . We assume this setting in the 
following. 

Lemma 2.4. Let T : [N] x [N] — > [M] be an (S, M) -balanced table. Let v be a string 
with \v\ < m. Then for all sets B\ C [N],B 2 C [N] with \B%\ > S, \B 2 \ > S, the number of 
entries in the B\ x B 2 rectangle ofT that have a color whose prefix is v is < 2 • • \B\ x B 2 \ . 

Proof. First observe that, since the table is (S, L>)-balanced with the value of the parameter 
D equal to M, the definition of an (S, L>)-balanced table implies that no color a € [M] 
occurs more than a fraction of 2/M times in any rectangle of T with sizes > S. Let v be a 
string of length of most m. Then v has 2 m ~^ extensions of length m and, as we have just 
noted, each such extension occurs at most a fraction 2/M in any rectangle with sizes > S. 
It follows that in any B\ x B 2 rectangle of T, all the extensions of v taken together occur 
at most 2 m '\ v \ ■ (2/M) ■ \B X x B 2 \ = (2/2\ v \) ■ \B X x B 2 \ times. ■ 
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3. Increasing the randomness rate of strings 

The next theorem shows that from two n-bit strings with complexity an and dependency 
an, one can construct a string of length ~ 2 cm and complexity ~ (2a — a)n. 

Theorem 3.1. For every a > 0, for every < a < a, there is a computable function 
f : {0, 1}* x {0, 1}* — > {0, 1}* ; that, for every n, maps any pair of strings of length n into a 
string of length m = 2an — log n and has the following property: for every sufficiently large 
n, if (x, y) is a pair of strings with 

(1) \ x \ = \y\ = n, 

(2) K(x) > an, K(y) > an 

(3) (x, y) have dependency at most an, 
then 

K(f(x, y)) > (2a - a)n - 9 log n. 

Proof. Let us fix n and let N = 2 n ,m = 2an — \ogn, M = 2 m , S = 2 an ,d = are + 81ogn, and 
D = 2 d . Note that the requirements of Lemma 12.31 are satisfied and therefore there exists a 
table T : [N] x [N] — > [M] that is (S, -D)-balanced. By brute force, we find the smallest (in 
some canonical sense) such table T. Note that the table T can be described with logn+0(l) 
bits. We define f(x,y) to be T(x,y). Thus, let z = T(x,y) for some strings x and y of 
length n satisfying the requirements in the theorem hypothesis. For the sake of obtaining a 
contradiction, suppose that K(z) < (2a — a)n — 9 log n = m — d. Let t\ = K(x) ,ti = K(y). 
From the properties of x and y, t\ > an and > an. Let B\ = {u G {0, l} n | K(u) < ti}, 
B 2 = {v G {0, l} n | K(v) < t 2 } and A = {w G {0, l} m | K(w) <m-d}. We have \Bt\ < 
2 tl+1 , \B 2 \ < 2 i2+1 and \A\ < 2 m ~ d . We take B[ and B' 2 with |^| = 2* 1+1 , \B' 2 \ = 2* 2+1 , 
B\ C B[ and B 2 C B' 2 . Since the table T is (S, -D)-balanced, 

\T- l (A) n (Bi x B 2 )\ < (T- 1 ^) n (B{ x B' 2 )\ < 2 • Jjjl • \B' X x B' 2 \ 

<; 2 . 2 m ~ d — ■ 2* 1+1 • 2' 2+1 

< 2*i+*2-d+3 

Note that (x, y) G T^(A) n (Si x B 2 ) and that T~ l (A) n (B 1 x 5 2 ) can be enumerated if we 
are given t\,t 2 and n (from which (m — d) and a description of table T can be determined). 
Therefore xy can be described by the rank of (x, y) in the above enumeration and by 
information needed for performing that enumeration. Thus 

K(xy) <h+t 2 -d + 2(log h + log t 2 + log n) + 0(1) 
< t\ + t 2 — d + 7 log n. 

For the second inequality, we took into consideration that t\ < n and t 2 < n. On the other 
hand, since x and y have dependency bounded by an. 

K(xy) >t\ + t 2 — an. 

Keeping in mind that d = an + 81ogn, we have obtained a contradiction. ■ 

The next theorem shows from two n-bit strings with complexity s(n) and dependency 
a(n), one can construct a string of length m ~ s(n)/2 with complexity conditioned by any 
one of the input strings ~ m — a(n). 

Theorem 3.2. For every computable function s(n) verifying 61ogn < s(n) < n and every 
function a(n), there is a computable function f : {0, 1}* x {0, 1}* — > {0, 1}* that, for every 
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n, maps any pair of strings of length n into a string of length m = s(n)/2 — 71ogra and has 
the following property: for every sufficiently large n, if (x, y) is a pair of strings with 

(1) | x | = \y\ = n, 

(2) K(x)>s(n),K(y)>s(n), 

(3) (x, y) has dependency at most a{n) 
then 

K( f(x, y) | x) >m — a(n) — 11 log n, 
K(f(x, y) | y) > m — a{n) — 11 log n. 

Proof. We fix n and let N = 2 n ,m = s(n)/2 - 71ogn,M = 2 m ,S = 2 S ^/ 2 ,D = M,t = 
a(n) + 11 log n. The requirements of Lemma 12.31 are satisfied and therefore there exists a 
table T : [N] x [N] — > [M] that is (S, D)-balanced. By brute force, we find the smallest (in 
some canonical sense) such table T. The table T is determined by n and s(n), and, thus, 
can be described with log n + log s(n) + 0(1) bits. Note that, since D = M, it holds that for 
every color a € [M] and for every subsets B\ C [N], B2 Q [N] with \B\ \ > S, |i?2| > S, the 
number of occurrences of a in the B\ x Bi subrectangle of T is bounded by (2/M) ■ \B± xB2\- 
We define f(x,y) to be T(x,y). Thus, let z = T(x,y) for some strings x and y of 
length n satisfying the requirements in the theorem hypothesis. We need to show that 
K (z I x) and K(z \ y) are at least m — a{n) — 11 log n. We show this relation for K{z \ y) 
(the proof for K (z \ x) is similar). For the sake of obtaining a contradiction, suppose that 
K{z I y) < m — a(n) — 11 log n = m — t. Let t\ = K(x). Note that t\ > s(n). Let 
B = {u£ {0, 1}" I K(u) < ti}. Note that 2* 1+1 > \B\ > 2 S ^/ 2 = S. We say that a column 
u € [N] is bad for color a € [M] and B if the number of occurrences of a in the B x {u} 
subrectangle of T is greater that (2/M) • \B\ and we say that u is fed /or B if it is bad 
for some color a and B. For every a £ [M], the number of u's that are bad for a and B 
is < S (because T is (S, Unbalanced). Therefore, the number of it's that are bad for B is 
< M ■ S. Given ti and a description of the table T, one can enumerate the set of u's that 
are bad for B. This implies that any u that is bad for B can be described by its rank in 
this enumeration and the information needed to perform the enumeration. Therefore, if u 
is bad for B, 

K(u) < log(M • 5) + 2(log h + log n + log s(n)) + O(l) 

< m + s(n) /2 + 6 log n + 0(1) 

< s(n), 

provided n is large enough. Since K(y) > s(n), it follows that y is good for B. 

Let A = {w £ [M] I K (uu \ y) < m — t}. We have \A\ < 2 m ~ t and, by our assumption, 
z £ A. Let G be the subset of B of positions in the strip B x {y} of T having a color from 
A (formally, G = proj 1 (T _1 (^4) n {B x {y})) . Note that x is in G. Each color a occurs in 
the strip B x {y} at most (2/M) • |£>| (because y is good for B). Therefore the size of G is 
bounded by 

\A\ ■ (2/M) • \B\ < 2 m - 1 ■ (2/M) • 2* 1+1 < 2 ll - t+2 . 

Given y,ti,m — t and a description of the table T, one can enumerate the set G. Therefore, x 
can be described by its rank in this enumeration and by the information needed to perform 
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the enumeration. It follows that 

K(x \y) < t x - t + 2 + 2 (log h + log(m - t) + log n + log s(ra)) + 0(1) 
< *! -t + 81ogn + 0(l) 
= t x - a(n) - 31ogn + 0(1) 
= K(x) - a(n) - 31og n + 0(1). 

Since K(xy) < K(y) + K(x \ y) + 2 logn + 0(1) (this holds for every n-bit strings x and y), 
we obtain 

K(xy) < K(y) + K(x) - a(n) - 3 log n + 2 log n + O(l) 
< if (y) + K(x) - a(n) - logn + 0(1), 
which contradicts that x and y have dependency at most a(n). m 

4. Increasing the randomness rate of sequences 

We prove that the randomness rate of sequences can be effectively increased even from 
two sequences having linear dependence. 

Theorem 4.1. There exists a truth-table reduction f with the following property. For any 
rational numbers r > and 5 > 0, there exists a > such that for any sequences x and y 
with randomness rate at least r and dependency at most an, f(x,y,T,5) has randomness 
rate at least 1 — 5. Moreover, the reduction f is uniform in x,y,r and 5. 
Proof. The plan is as follows. We split x into strings x\X2 ■ ■ ■ X{ . . . and y into strings 

2/12/2 • • • Hi For each i, let Xj = xx ■ ■ ■ X{ and y^ = yi ■ ■ ■ y%. The splitting is done in 

such a way that xi and yi have complexity close to r\xi\ and respectively close to r\yi\ 
even conditioned by x^xVi-x- Next, for each i, we construct a balanced table Tj with 
appropriate parameters and take z,- L = T(xi,yi). The output of the truth-table reduction 
is the sequence z = z\Zi ■ ■ ■ Zi . . .. As in the case of strings, it follows that Zi has high 
complexity and actually this holds even conditioned by = ZxZ2 ■ ■ ■ So far, the 

proof is as in [Zim 08] . The point of departure is that in order for the construction to work 
with inputs having linear dependence, we need to take the length of Zi exponential in i 
(rather than quadratic in i, which was the case in [Zim08]). This creates difficulties in 
showing that every "intermediate" prefix of z (i.e., a string that is an extension of Hi—\ and 
a prefix of Zi, for some i) has high complexity. To handle this, we argue that even prefixes 
of Zi have relatively high Kolmogorov complexity conditioned by Xi-xVi-x ( see Lemma |4.3|) 
and then the argument for "intermediate" strings forks into two cases depending on whether 
the string is long or short (see Lemma l4.4p . 
We proceed with the formal proof. 

We fix rational numbers r > and 5 > 0. Let x and y be sequences with randomness 
rate at least r. Let e = 5/4. 

We split x = x\X2 ■ ■ ■ Xi . . . and y = yxVi ■ ■ ■ Ui ■ ■ ■ and let rii = \xi\ = \yi\. We'll take 
ni = B % for some constant B, given by the next lemma. 

Lemma 4.2. There exists a constant B > 1 with the following properties: 

(a) For every i, K(xi \ Xi-xVi-x) — 0.99mj and K{yi \ Xi-xVi-x) — 0.99mj. 

(b) For any a > 0, if (x, y) have dependency an, then, for all i 

K(xiyi | Xi-xVi-x) > K ( x i I Xi-Wi-x) + K (Vi I #i-i2/i-i) ~ (2- 1 ) ■ a ■ n { . 
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Proof. (Sketch.) The proof is similar to an analogous result from [Zim08] . For (a), it is 
easy to show that B can be taken large enough so that the length of X{ is so much larger 
than the length of x~i-iy i _ 1 that the complexity of X{ does not decrease too much if it is 
conditioned by x~i^iy i _ 1 . 

The proof of (b) passes through the following intermediate steps: 

(1) We show that for B, i and j sufficiently large, 

Kfaxj) = K{yi) + K(xj) ± 1.001a(S' + B j ). 

(This is the analogue of Lemma 4.5 from [Zim08j). 

(2) We show that B, i and j sufficiently large, 

K( Xi | Xi-iVj) = K{ Xi | Xi-x) ± 2.004a( J B i + B*). 

(This is the analogue of Lemma 4.6 from [Zim08j; the constants are not optimized). 

Next, the statement can be shown similarly to Lemma 4.7 from |Zim08]). ■ 

For the rest of this section, we fix the following parameters as follows: 

• The constant B is as given by Lemma l4.2i 

• a = (l/3)e 2 • (0.97r) • (l/B). 

• For each i, N t = T\ S t = 2(°- 98r ) % m, = (0.97t) • n h M, L = 2 m * D { = Mi. 

The parameters satisfy the requirements of Lemma 12.31 and, thus, for each i, there exists a 
table Tj : [Ni\ x [iVj] — > [Mi] that is (Si, Dj)-balanced. For every i, given i, a smallest (in 
some canonical sense) such table Tj can be constructed by exhaustive search. We fix these 
tables Ti and define Zi = Ti(xi,yi) and next z = z\z 2 ■ ■ ■ z\ . . .. Clearly z is constructed by a 
truth-table reduction / from input sequences x and y. We will show that z has randomness 
rate at least 1 — 5. 

Lemma 4.3. For every i sufficiently large, each prefix v of z% has K(v \ Xj_iy i _ 1 ) > 
\v\ — 3a • rii- 

Proof. Suppose that there is a prefix v of z,- t with K(v \ Xi-iy i _ 1 ) < \v \ — 3a ■ rij. We define: 

• h = K(xi | Xi-iy^x), t 2 = K(yi \ Xi-iy^x), 

• B 1 = {ue {0, l} n * I K(u | Xi-iVi-x) < h}, 

• B 2 = {ue {0, 1}^ | K(u I Xi-w^) < h}, 

• A = {w G {0, l}' 1 '! | K(w | Xi-iy~i_i) < \v\ — 3a ■ rii}. 

Note that t% > 0.99r • n u t 2 > 0.99r • n { (by Lemma H2J), 2' 1+1 > \B X \ > 2°- 98r ni = Si, 
2*2+1 > > 2°- 98t ^ = Si and \A\ < 2^- 3an \ Let G be the set of entries (represented 
by their coordinates in the table) in the B\ x B 2 rectangle of the table Tj that have a color 
with a prefix in A. By Lemma 12.41 the cardinality of G is at most 

2 • ^ ■ \Bi x B 2 \ < 2 ■ 2\ v \~ 3an * ■ ^ ■ 2* 1+1 • 2 t2+1 

_ 2*l+*2— 3arij+3 

Note that (xi,yi) belongs to G and that G can be enumerated given Xi-iy i _ 1 , ti,t 2 , and 
\v\ — 3a ■ rii (observe that i can be determined from Xi-iy^i and thus the table Tj can be 
constructed). Therefore x%\ji can be described by its rank in the enumeration of G and by 
the information needed to perform this enumeration. This implies 

K(xiyi I Wi-iy^) < t\ + t 2 - 3a ■ rii + 2(logti + logt 2 + log(|v| - 3anj)) + 0(1) 
< ti + t 2 - 3a ■ rii + O(lognj). 
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On the other hand, by Lemma 14.21 

Kfayi | Xi-iy^) > K(xi | Xi-xy^x) + K(y { \ Xi-iy^) - (2.1) • a ■ n*. 

If i is large, the last two inequalities conflict each other and we obtain a contradiction. ■ 

The next lemma finishes the proof of Theorem 14.11 

Lemma 4.4. For each sufficiently long prefix uu of z, K(w) > (1 — 4e)|w|. 

Proof. For some i, the prefix w is of the form w = z\ . . . Zi_\Vi, with Vi a prefix of Zj. Let 
7 = (1/e) • (3a). We consider two cases: 

Case 1: Vi is long. Suppose \vi\ > 7 • rtj. 

Then K(yi | Xj_iy i _ 1 ) > \vi\ — 3a • > \v.- L \ — (3a/7) • \v-- L \ = (1 — e)|«j|. This implies 
i^(uj I z\ . . . Zi-i) > (1 — e) • \vi\ — 0(1) > (1 — 2e)|*yj|, because each 2^ can be constructed 
from Xj and yj. By induction, it follows that K(z\Z2 ■ ■ ■ ^-it^) > (1 — 2>e)\z\Z2 ■ ■ ■ 
For the induction step, the argument goes as follows: 

K(z\z 2 . . . Zi-iVi) > K(zi . . . + K(vi I Z\ . . . Zi-\) 

-0(log(mi + . . . + m,_i) + log(|uj|)) 

> (1 - 3e)(mi + . . . + mi-i) + (1 - 2e)|wi| 

-0(log(mi + . . . + m,_i) + log I^D) 

> (1 - 3e)(m 1 + . . . + mj_i + \vi\). 

In the last step, we have used the fact that log(mi + . . . + mj_i) = 0(i), log = 0(i) and 

\ Vi \ = 

Case 2: -Uj is short. Suppose |t>i[ < 7 • n^. 

For a contradiction, suppose -ftT(ziZ2 • • • ^i-i^i) < (1 — ^)\z\Z2 ■ ■ ■ z,i_\Vi\. Note that 
z\Z2 ■ ■ ■ Zi_\ can be reconstructed from a descriptor of z\Zi ■ ■ ■ Zi_\V{. This implies 

K(z 1 z 2 ■ ■ ■ Zi-i) < (1 - 4e)(mi + m 2 + . . . + m^i + \vi\) + O(l) 
= (1 - 4e)(mi + . . . + rm-i) + (1 - 4e)|^| + 0(1) 

< (1 - 4e)(mi + . . . + rm-i) + (1 - 4e)7 ■ 

< (1 - 4e)(mi + . . . + mi_i) + (1 - 4e) • (l/e)(3a) • n^. 

But the second term is less than e{m\ + . . . + w,?.-i) (due to the choice of a). This implies 
that K(z\Z2 ■ ■ ■ < (1 — 3e)(mi + m<i + . . . + m;_i), which, by Case 1, is not possible. ■ 

■ 

Note. It remains an open issue whether from input sequences x and y (even indepen- 
dent) one can construct a sequence z that has high randomness rate conditioned by any 
one of the input sequences. In other words, the infinite analogue of Theorem 13.21 is open. 
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